Stereo image processing apparatus and stereo image processing method

ABSTRACT

Provided is a stereo image processing apparatus wherein parallax can be calculated with high precision. A window-function shifting unit ( 411 ) sets a third window function, which is formed by shifting a second window function on the basis of the amount of deviation in sub-pixel units, onto an image cutting-out unit ( 412 ). The image cutting-out unit ( 412 ) applies the second window function to a position subjected to a cut-out, cuts out a unit partial target image from a standard image, applies the second window function or the third window function, and cuts out a unit partial reference image from a reference image. A peak-position detection unit ( 106 ) calculates the amount of deviation in sub-pixel units on the basis of the phase difference between a data string comprising brightness of the cut-out unit partial target image, and a data string comprising brightness of the cut-out unit partial reference image.

TECHNICAL FIELD

The claimed invention relates to a stereo image processing apparatus anda stereo image processing method that compute, based on stereo images (atarget image and a reference image) capturing the same object, thedisparity between the images caused by parallax.

BACKGROUND ART

There are conventionally known stereo image processing apparatuses thatcompute, based on two images (a target image and a reference image) ofthe same object as captured by a stereo camera, the disparity betweenthose images, and that measure the distance to the object based on thecomputed disparity between the images. Applications considered for thesestereo image processing apparatuses include, for example, an apparatusthat measures the distance to a vehicle or pedestrian ahead based onstereo images of the vehicle or pedestrian taken by a vehicle-mountedcamera. However, due to the reduction in the sizes of cameras (e.g.,vehicle-mounted cameras) in recent years, camera separations are alsobecoming smaller, as a result of which disparities between stereo imagesare also becoming smaller. Accordingly, accurate disparity computationfunctionality is beginning to be demanded of stereo image processingapparatuses.

As an accurate stereo matching (disparity computation in stereo imageprocessing) scheme for stereo image processing apparatuses, aone-dimensional phase only correlation (POC) scheme has been proposed,for example (see Patent Literature 1, for example). In thisone-dimensional POC scheme, a partial image (a one-dimensional imagedata sequence) is first extracted from each of a target image and areference image using the Hanning window. Next, the extracted partialtarget image and partial reference image undergo a one-dimensionalFourier transform to be turned into Fourier image data, and aresubsequently combined. The amplitude components of the combined Fourierimage data are normalized, after which a one-dimensional inverse Fouriertransform is performed. A phase-only correlation coefficients are thusderived. The disparity between the images (parallax) is then computedbased on the correlation peak of the phase-only correlationcoefficients.

CITATION LIST Patent Literature

PTL 1

Japanese Patent Application Laid-Open No. 2008-123141

SUMMARY OF INVENTION Technical Problem

However, the related art above has a problem in that it is difficult toaccurately compute parallax for objects that occupy a small image regionsize in stereo images in the base line length direction (hereinafterreferred to as the “image region size in the base line lengthdirection”), examples of which include pedestrians located afar, and soforth. This is because, although, when the image region size in the baseline length direction is small, the one-dimensional image data sequenceneeds to be made small in order to reduce the influence of the image ofthe background/surroundings, the accuracy of the correlation peakmentioned above decreases as the one-dimensional image data sequencebecomes smaller.

The claimed invention is made in view of the point above, and an objectthereof is to provide a stereo image processing apparatus and a stereoimage processing method that enable accurate disparity computation evenfor objects of small image region sizes in the base line lengthdirection.

Solution to Problem

A stereo image processing apparatus according to an embodiment of theclaimed invention is a stereo image processing apparatus that computes asubpixel-level disparity between a target image and a reference imagethat form stereo images, the stereo image processing apparatusincluding: an extraction section that extracts a unit partial targetimage for subpixel estimation from the target image by applying a firstwindow function at an extraction target position, and that extracts aunit partial reference image for subpixel estimation from the referenceimage by applying the first window function or a second window functionthat is set; a computation section that computes the subpixel-leveldisparity based on a phase difference between a data sequence includingintensity values of the extracted unit partial target image for subpixelestimation and a data sequence including intensity values of theextracted unit partial reference image for subpixel estimation; and awindow function setting section that sets, with respect to theextraction section, the second window function, which is formed byshifting the first window function based on the subpixel-level disparitycomputed by the computation section.

A stereo image processing method according to an embodiment of theclaimed invention is a stereo image processing method that computes asubpixel-level disparity between a target image and a reference imagethat form stereo images, the stereo image processing method including:an extraction step of extracting a unit partial target image forsubpixel estimation from the target image by applying a first windowfunction at an extraction target position, and extracting a unit partialreference image for subpixel estimation from the reference image byapplying the first window function or a second window function that isset; a computation step of computing the subpixel-level disparity basedon a phase difference between a data sequence including intensity valuesof the extracted unit partial target image for subpixel estimation, anda data sequence including intensity values of the extracted unit partialreference image for subpixel estimation; and a window function settingstep of forming and setting the second window function by shifting thefirst window function based on the subpixel-level disparity computedthrough the computation step.

Advantageous Effects of Invention

With the claimed invention, it is possible to provide a stereo imageprocessing apparatus and a stereo image processing method that enableaccurate parallax computation even for objects of small image regionsizes in the base line length direction.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the configuration of a stereo imageprocessing apparatus according to Embodiment 1 of the claimed invention;

FIG. 2 is a block diagram showing the configuration of a filter section;

FIG. 3 is a block diagram showing the configuration of a high-accuracyfilter section;

FIG. 4 is a flow chart illustrating operations of a stereo imageprocessing apparatus;

FIG. 5 is a diagram illustrating a process by an image matching section;

FIG. 6 is a flow chart showing the details of a subpixel-levelcomputation process;

FIG. 7 is a diagram illustrating the concept of a subpixel-levelcomputation process;

FIG. 8 is a diagram illustrating the concept of a subpixel-levelcomputation process;

FIGS. 9A through 9E are charts illustrating the concept of a process bya window shift section;

FIG. 10 is a chart illustrating the concept of a process by a windowshift section;

FIG. 11 is a block diagram showing the configuration of a stereo imageprocessing apparatus according to Embodiment 2 of the claimed invention;and

FIG. 12 is a flow chart illustrating operations of a stereo imageprocessing apparatus.

DESCRIPTION OF EMBODIMENTS

Embodiments of the claimed invention are described in detail below withreference to the drawings. With regard to the embodiments, like elementsare designated with like reference numerals, while descriptions thereofare omitted to avoid redundancy. In the descriptions below, it isassumed that the X-axis lies in the horizontal direction of the image,that the Y-axis lies in the vertical direction of the image, and thatone pixel represents one coordinate point.

Embodiment 1

[Configuration of Stereo Image Processing Apparatus 100]

FIG. 1 shows a configuration of stereo image processing apparatus 100according to Embodiment 1 of the claimed invention. With respect to FIG.1, stereo image processing apparatus 100 includes stereo imageacquisition section 101, image matching section 102, filter section 103,peak position detection sections 104 and 106, and high-accuracy filtersection 105.

Stereo image acquisition section 101 acquires stereo images taken withtwo or more imaging systems (i.e., cameras). The stereo images include atarget image and a reference image of the same object taken by twodistinct imaging systems. Stereo image acquisition section 101 thenoutputs the acquired stereo images to image matching section 102, filtersection 103, and high-accuracy filter section 105. For the presentembodiment, it is assumed that stereo image acquisition section 101acquires stereo images taken by two cameras whose base line lengthdirection generally coincides with the horizontal direction.

Image matching section 102 acquires, on a pixel level, a point in thereference image corresponding to a target point in the target image.Specifically, image matching section 102 acquires, on a pixel level, apoint the reference image corresponding to a target point in the targetimage by performing an image matching process on the target image andreference image acquired at stereo image acquisition section 101. Imagematching section 102 then computes “pixel-level disparity n” between thetarget image and the reference image. On the other hand, the “parallax”between the target image and the reference image obtained by peakposition detection section 104 and peak position detection section 106,which are discussed hereinafter, is given in subpixels. In other words,the disparity between the target image and the reference image isroughly detected in “pixels” at image matching section 102, and,thereafter, the disparity between the target image and the referenceimage is finely detected in “subpixels” by peak position detectionsection 104 and peak position detection section 106.

Specifically, image matching section 102 takes one predetermined pixelincluded in the target image to be a “target point,” and extracts fromthe target image a partial image of the surroundings centered around thetarget point (hereinafter referred to as a “unit partial target image”).Image matching section 102 also extracts from the reference image aplurality of partial images of the same size as the unit partial targetimage (hereinafter referred to as “unit partial reference images”). Theplurality of unit partial reference images are extracted from thereference image at different positions. In extracting the unit partialtarget image and the unit partial reference images, a rectangular windowof a predetermined size (vertical size: wv pixels, horizontal size: whpixels) may be used, for example. The window function that defines thisrectangular window will be referred to below as a “window function forpixel-level estimation.”

In the case of stereo cameras, parallax between the target image and thereference image only occurs in the base line length direction of thecamera. Accordingly, image matching section 102 may extract theplurality of unit partial reference images by varying the extractionposition along the base tine length direction. Furthermore, thedisparity between the position of the target point in the target imageand the position of the corresponding point in the reference image iscomputed as pixel-level disparity n mentioned above.

Image matching section 102 identifies, from among the plurality ofextracted unit partial reference images, the unit partial referenceimage with the greatest level of match with respect to the unit partialtarget image. The one pixel in the thus identified unit partialreference image corresponding to the “target point” is the “pixel-levelcorresponding point” in the reference image. As a measure of the levelof match, a sum of absolute differences (SAD) value, which signifiesintensity difference, may be used, for example.

Filter section 103 takes, as input, the target point and disparity nfrom image matching section 102, as well as the stereo images fromstereo image acquisition section 101.

Filter section 103 then computes an inverted phase filter coefficientbased on the target image and the position of the target point. Filtersection 103 then performs a filtering process around the pixel-levelcorresponding point in the reference image using the thus computedinverted phase filter coefficient.

FIG. 2 shows a configuration of filter section 103. With respect to FIG.2, filter section 103 includes image extraction section 402 andfiltering section 403.

Image extraction section 402 extracts, as a unit partial target imagefor subpixel estimation, a partial image from the target image in thebase line length direction of the stereo images. A window function forsubpixel-level estimation is used for the extraction of the unit partialtarget image for subpixel estimation. The Hanning window function may beused for the window function for subpixel-level estimation, for example.Furthermore, image extraction section 402 uses the window function forsubpixel-level estimation to extract, from the reference image and as aunit partial reference image for subpixel estimation, a partial image ofthe same size as the unit partial target image for subpixel estimation.

Image extraction section 402 then outputs to filtering section 403 theunit partial target image for subpixel estimation and the unit partialreference image for subpixel estimation.

For the present embodiment, it is assumed that image extraction section402 determines the image extraction position in the target image in sucha manner that the target point would be included in the unit partialtarget image for subpixel estimation. Furthermore, it is assumed thatimage extraction section 402 determines the image extraction position insuch a manner that the pixel-level corresponding point would be includedin the unit partial reference image for subpixel estimation.

Filtering section 403 computes an inverted phase filter coefficient,which is obtained by reversing, in the front-back direction, theposition of each pixel value in the unit partial target image forsubpixel estimation extracted by image extraction section 402. Using thethus computed inverted phase filter coefficient, filtering section 403then performs a filtering process on the unit partial reference imagefor subpixel estimation. Filter generation section 403 then outputs theresults of the filtering process (hereinafter referred to as “filteringresults”) to peak position detection section 104.

With respect to the filtering results received from filter section 103,peak position detection section 104 acquires the relative positionalrelationship that provides the highest level of correlation between theunit partial target image for subpixel estimation and the unit partialreference image for subpixel estimation. Peak position detection section104 then computes, based on the acquired relative positionalrelationship, the parallax (disparity) between the unit partial targetimage for subpixel estimation and the subpixel-level corresponding pointin the unit partial reference image for subpixel estimation with respectto the target point.

High-accuracy filter section 105 takes, as input, the target point anddisparity n from image matching section 102, and the stereo images fromstereo image acquisition section 101, as well as the subpixel-leveldisparity from peak position detection section 104.

Like filter section 103, high-accuracy filter section 105 computes aninverted phase filter coefficient based on the unit partial target imagefor subpixel estimation, and performs filtering on the unit partialreference image for subpixel estimation using the thus computed invertedphase filter coefficient. In other words, like filter section 103,high-accuracy filter section 105 first extracts the unit partial targetimage for subpixel estimation from the target image, and computes theinverted phase filter coefficient based on the unit partial target imagefor subpixel estimation. In other words, the inverted phase filtercoefficient is computed based on the unit partial target image forsubpixel estimation extracted from the target image.

However, unlike filter section 103, high-accuracy filter section 105next shifts, by an amount corresponding to the subpixel-level disparityreceived from peak position detection section 104, the window functionfor subpixel-level estimation that was used to extract the unit partialreference image for subpixel estimation at filter section 103, andthereby forms a shifted window function for subpixel-level estimation tobe used in extracting the unit partial reference image for subpixelestimation at high-accuracy filter section 105. Using the shifted windowfunction for subpixel-level estimation, high-accuracy filter section 105then extracts the unit partial reference image for subpixel estimationfrom the reference image. Using the computed inverted phase filtercoefficient, high-accuracy filter section 105 performs a filteringprocess on the unit partial reference image for subpixel estimation, andoutputs the filtering results to peak position detection section 106.

FIG. 3 shows a configuration of high-accuracy filter section 105. Withrespect to FIG. 3, high-accuracy filter section 105 includes windowfunction shift section 411, image extraction section 412, and filteringsection 413.

By shifting, by an amount corresponding to the subpixel-level disparitycomputed at peak position detection section 104, the window function forsubpixel-level estimation used in extracting the unit partial referenceimage for subpixel estimation at filter section 103, window functionshift section 411 forms the shifted window function for subpixel-levelestimation to be used in extracting the unit partial reference image forsubpixel estimation at high-accuracy filter section 105.

Image extraction section 412 extracts from the target image the unitpartial target image for subpixel estimation including the target point.Furthermore, using the shifted window function for subpixel-levelestimation formed at window function shift section 411, image extractionsection 412 extracts from the reference image the unit partial referenceimage for subpixel estimation including the corresponding point (i.e., apoint in the reference image that is displaced by disparity n from thesame coordinates as the target point). In other words, image extractionsection 412 has generally the same functionality as image extractionsection 402.

Filtering section 413 computes an inverted phase filter coefficientbased on the unit partial target image for subpixel estimation extractedat image extraction section 412. Using the computed inverted phasefilter coefficient, filtering section 413 then performs a filteringprocess on the unit partial reference image for subpixel estimation, andoutputs the filtering results to peak position detection section 106. Inother words, filter section 413 has generally the same functionality asfiltering section 403.

Referring back to FIG. 1, peak position detection section 106 computesthe subpixel-level disparity between the target image and the referenceimage by detecting the position of the peak in the filtering resultsreceived front high-accuracy filter section 105. The term “peak” as usedabove refers to the position at which the filtering results exhibit thegreatest value.

[Operations of Stereo Image Processing Apparatus 100]

Operations of stereo image processing apparatus 100 configured thus aredescribed below.

In the descriptions below, it is assumed that the X-axis lies in thehorizontal direction of the image, that the Y-axis lies in the verticaldirection of the image, and that one pixel represents one coordinatepoint. For purposes of convenience, it is also assumed that thedirection of epipolar lines (the base line length direction) is parallelto the X-axis across the entirety of the images.

FIG. 4 is a flow chart illustrating operations of stereo imageprocessing apparatus 100. Although what follows is a description of aprocess with regard to a given target point in the target image, stereoimage processing apparatus 100 performs the operations of steps S1through S9 below with respect to all pixels within a ranging targetregion by sequentially moving the target point across the entire targetimage.

<Analysis Target Position Determination Process>

In step S1, image matching section 102 determines the position of atarget point (hereinafter referred to as an “analysis target position”)which is to serve as a target for analysis in a ranging target region.

<Unit Partial Target Image Extraction Process>

In step S2, image matching section 102 extracts a unit partial targetimage from the target image received from stereo image acquisitionsection 101. The unit partial target image is a partial region image (animage region) based on the analysis target position. (i.e., the targetpoint) determined in step S1. The size of the unit partial target imageis expressed in pixels. In other words, the unit partial target image isan image made up of a plurality of pixel rows and a plurality of pixelcolumns.

<Search Range and Search Start Position Determination Process>

In step S3, image matching section 102 determines a search range and asearch start position with respect to the reference image based on theanalysis target position determined in step S1. The parallax of thestereo images is determined by the base line length, which is thedistance between the cameras, the focal length of the lenses, and thedistance from the stereo camera to the object of interest. Accordingly,image matching section 102 may determine the search range based on thedistance from the stereo camera to the object to be ranged. Furthermore,since an object that is at infinity from the stereo camera is imaged atthe same position in the target image and the reference image, imagematching section 102 may set the same coordinates as the target point inthe target image to be the search start position in the reference image.

<Unit Partial Reference Image Extraction Process>

In step S4, image matching section 102 determines an extraction targetposition, and extracts from the reference image a unit partial referenceimage of the same size as the unit partial target image. Image matchingsection 102 may take the search start position determined in step S3 tobe the initial extraction target position, for example, and subsequentlymove the extraction target position.

<Matching Score Computation Process>

In step S5, image matching section 102 computes the level of matchbetween the unit partial target image and the unit partial referenceimages. For this level of match, an SAD value, which is a measure ofintensity difference, or intensity similarity may be used, for example.

<Search Range Completion Determination Process>

In step S6, image matching section 102 performs a completiondetermination process regarding the processing of the search range. Inother words, image matching section 102 determines whether or not thesearch range has been covered by moving the extraction target position.If it is determined that the search range has not been completed (stepS6: No), image matching section 102 returns to step S4. As a result,image matching section 102 moves the extraction target position withinthe search range so as to displace the extraction region for the unitpartial reference image in step S4. Thus, the processes of steps S4through S6 are repeated until the search range is completed (step S6:Yes).

<Matching Score Maximum Position>

In step S7, based on the plurality of matching scores obtained throughthe processes of steps S4 through S6, image matching section 102determines the position of the unit partial reference image at which thematching score becomes greatest. If intensity difference is used for thematching score, image matching section 102 detects the unit partialreference image that results in an extremely small, or the smallest,intensity difference.

The processes of steps S2 through S7 will now be described in detailwith reference to FIG. 5.

As shown in FIG. 5, in step S2 mentioned above, image matching section102 extracts as a unit partial target image a partial image of thesurroundings centered around analysis target position (target point)(xa, ya). For this extraction of a unit partial target image, arectangular window of a predetermined size (vertical size: wv pixels,horizontal size: wh pixels) defined by a window function for pixel-levelestimation is used. Furthermore, although it is assumed in thedescription below that the center of the rectangular window defined bythe window function for pixel-level estimation is aligned with theanalysis target position, it need not be strictly at the center, and theanalysis target position need only be located near the center of therectangular window.

Next, in step S3 mentioned above, image matching section 102 determinesa search range and a search start position in the reference image basedon the analysis target position determined in step S1. For the searchstart position (the initial coordinates for extracting a unit partialreference image from the reference image), the same coordinates (xa, ya)as the analysis target position in the target image may be used, forexample. Next, as shown in FIG. 5, image matching section 102 extracts,from the reference image and as unit partial reference images, partialimages surrounding the search start position at the center in step S4while sequentially shifting the extraction target position. For the caseat hand, the extraction target position is shifted by one pixel at atime. For this extraction of unit partial reference images, the samerectangular window as the rectangular window used for the extraction ofthe unit partial target image is used. In other words, for theextraction of the unit partial target image and the unit partialreference images, rectangular windows defined by the same windowfunction for pixel-level estimation are used.

In step S5 mentioned above, image matching section 102 then computes thelevel of match between the unit partial target image and each unitpartial reference image. For this level of match, an SAD value, which isa measure of intensity difference, may be used, for example. This SADvalue is computed through equation 1 below.

$\begin{matrix}{\left( {{Equation}\mspace{14mu} 1} \right)\mspace{619mu}} & \; \\{{S\; A\; {D(n)}} = {\overset{{y\; a} + {{wv}/2}}{\sum\limits_{j = {{y\; a} - {{wv}/2}}}}{\overset{{x\; a} + {{wh}/2}}{\sum\limits_{i = {{x\; a} - {{wh}/2}}}}{{{f\left( {{x + i},{y + j}} \right)} - {g\left( {{x + i + n},{y + j}} \right)}}}}}} & \lbrack 1\rbrack\end{matrix}$

Then, as shown in FIG. 5, if, in step S6 mentioned above it isdetermined that the search range has not been completed, image matchingsection 102 shifts the extraction position, returns to step S4, andextracts a new unit partial reference image from the reference image.For the case at hand, the extraction position is shifted by one pixel ata time. Furthermore, the direction of the shift is the directionindicated by the right arrow extending from coordinates (xa, ya) in thereference image in FIG. 5.

Image matching section 102 thus computes, with respect to a single unitpartial target image, the respective SAD values of a plurality of unitpartial reference images. In step S7, based on the plurality of matchingscores obtained through the process of step S5, image matching section102 then identifies the unit partial reference image that has thegreatest matching score. Specifically, image matching section 102identifies the unit partial reference image corresponding to thesmallest SAD value among the plurality of SAD values, for example.Assuming that the coordinates of the extraction target position for thethus identified unit partial reference image is (xa+n, ya), then n isthe disparity in pixels. Image matching section 102 then takes theextraction target position at which the SAD value is smallest to be thepixel-level corresponding point for the analysis target position (targetpoint). Although SAD values have been used above as measures of matchingscore, the claimed invention is by no means limited as such, andanything that may be used as a measure of matching score may be employedinstead. By way of example, image matching section 102 may also use thesum of squared differences (SSD) as a measure of matching score.

The above concludes this detailed description of the processes of stepsS2 through S7.

<Subpixel-Level Computation Process>

In step S8 in FIG. 4, filter section 103 and peak position detectionsection 104 perform a subpixel-level computation process based on thepixel-level corresponding point obtained in step S7 and on the targetimage and reference image received from stereo image acquisition section101.

FIG. 6 is a flow chart showing the details of a subpixel-levelcomputation process. FIG. 7 is a diagram illustrating the concept of asubpixel-level computation process.

(Extraction Process for Unit Partial Target Image for SubpixelEstimation)

In step S12, image extraction section 402 extracts from the unit partialtarget image a unit partial target image for subpixel estimation. Thesize of the unit partial target image for subpixel estimation is givenin pixels. The extraction position for the unit partial target image forsubpixel estimation is assumed to be from the position of the targetpoint, and its direction is assumed to he the X-axis direction, which isparallel to an epipolar line.

Furthermore, for the extraction of the unit partial target image forsubpixel estimation, a window function for subpixel-level estimation isused. As the window function for subpixel-level estimation, windowfunction w(m) of the Hanning window represented by equation 2 may beused, for example.

$\begin{matrix}{\left( {{Equation}\mspace{14mu} 2} \right)\mspace{619mu}} & \; \\{{w(m)} = {\frac{1}{2}\left\{ {{\cos \left( {\pi \frac{m}{K - J}} \right)} + 1} \right\}}} & \lbrack 2\rbrack\end{matrix}$

Although the description below involves a case where a Hanning windowfunction is used as the window function for subpixel-level estimation,the claimed invention is by no means limited as such, and it is alsopossible to use the Hamming window, the Blackman window, the Kaiserwindow, and/or the like, for the window function. A choice is made fromamong these window functions in accordance with which property of theunit partial target image for subpixel estimation is to be prioritized(e.g., spectral power property, phase property, extraction edgecontinuity). By way of example, if the phase property is to beprioritized, the Kaiser window would be suitable. However, using theKaiser window results in very complex computations. On the other hand,from the perspective of reducing computations, the Hanning window isfavorable.

In the image extraction process for estimating parallax on a subpixellevel, it is important that the extracted image be free of noise. Thisis to ensure that disparity is derived accurately on a subpixel level.On the other hand, since the image extraction process at image matchingsection 102 is performed on a pixel level, reducing the number ofcomputations is given greater importance than is accuracy. Accordingly,for the window function for pixel-level estimation used at imagematching section 102, a window function for merely extracting image datais used. By contrast, since noise being low holds greater importance forthe window function for subpixel-level estimation used in the extractionprocess for the unit partial target image for subpixel estimation, it ispreferable that it be a function where, as compared to the windowfunction for pixel-level estimation, changes at both ends of the windoware continuous (i.e., a function where the values at the beginning andend of one period are zero).

By using such a window function for subpixel-level estimation,continuity is ensured for the signal of the unit partial target imagefor subpixel estimation, and noise components caused by extraction,which are included in the properties of the later-described invertedphase filter, may be reduced. Comparing the window function forpixel-level estimation and the window function for subpixel-levelestimation with regard to their frequency properties, the windowfunction for pixel-level estimation has a main-lobe of a narrower widthand a side-lobe of a greater amplitude as compared to the windowfunction for subpixel-level estimation.

In FIG. 7, for window function w(m) for subpixel-level estimation, aHanning window so sized that the vertical axis is one pixel and thehorizontal axis is “K-J” pixels is used. m is an integer equal to orgreater than J but equal to or less than K. Window function w(m) forsubpixel-level estimation is set with target point (xa, ya) at thecenter. Thus, an image whose vertical axis size is one pixel and whosehorizontal axis size is “K-J” pixels is extracted, as the unit partialtarget image for subpixel estimation, with target point (xa, ya) at thecenter. f′(m) represents the intensity value of the unit partial targetimage for subpixel estimation.

(Extraction Process for Unit Partial Reference Image for SubpixelEstimation)

In step S13, image extraction section 402 extracts a unit partialreference image for subpixel estimation from the unit partial referenceimage detected in step S7 which has the greatest level of match withrespect to the unit partial target image. The same window function forsubpixel-level estimation as that for the unit partial target image forsubpixel estimation is also used in the extraction process for the unitpartial reference image for subpixel estimation. However, the windowfunction for subpixel-level estimation is set at corresponding point(xa+n, ya). Thus, an image whose vertical axis size is one pixel andwhose horizontal axis size is “K-J” pixels is extracted, as a unitpartial reference image for subpixel estimation, with correspondingpoint (xa+n, ya) at the center. In FIG. 7, g′(m) represents theintensity value of the unit partial reference image for subpixelestimation.

In the description above, window function w(m) for subpixel-levelestimation so sized that the vertical axis is one pixel and thehorizontal axis is “K-J” pixels is used. However, this size is merely anexample, and is by no means limiting. By way of example, if the verticalsize is taken to be three pixels, the process described above may beperformed one pixel at a time, and the results thus obtained may beaveraged. Furthermore, by way of example, if the vertical size spans aplurality of pixels, the process described above may be performed onevertical pixel at a time, and the results per pixel included in thevertical size may be weighted and averaged. The weighting coefficientsto be used in this case may be determined by a window function as intwo-dimensional POC. If the vertical size of the unit partial targetimage for subpixel estimation and the unit partial reference image forsubpixel estimation is two pixels or greater, image extraction section402 performs an averaging process such as that above, and converts eachof the unit partial target image for subpixel estimation and the unitpartial reference image for subpixel estimation into a one-dimensionaldata sequence.

(Inverted Phase Filter Coefficient Computation Process)

In step S14, filtering section 403 computes an inverted phase filtercoefficient based on the unit partial target image for subpixelestimation. Specifically, filter section 103 rearranges the datasequence in reverse order by inverting the positions of the pixels inthe constituent data sequence of the unit partial target image forsubpixel estimation.

(Filtering Process)

In step S15, filtering section 403 performs filtering on the unitpartial reference image for subpixel estimation using the inverted phasefilter coefficient computed in step S14, and outputs the filteringresults to peak position detection section 104.

When determining parallax with respect to an object of the same size ina real space, if the object is located far from the stereo camera,parallax would be less as compared to when it is located closer, andpixel-level disparity n would also be smaller. At the same time, theimage region size for the object in the base line length direction wouldalso be smaller.

Accordingly, it is preferable that the tap length of the inverted phasefilter be set in accordance with the size of pixel-level disparity ndetected at image matching section 102. By way of example, ifpixel-level disparity n is small, the tap length of the inverted phasefilter is set short accordingly. By adaptively varying the size of theunit partial target image for subpixel estimation and the unit partialreference image for subpixel estimation with respect to disparity n,stereo image processing apparatus 100 is able to adaptively vary the taplength of the inverted phase filter as well. Parallax computationcommensurate with the size of the object of interest thus becomespossible.

The filtering results are output of a linear shift invariant system.Accordingly, once lens distortion correction errors, errors such as gainnoise and/or the like stemming from image sensors (e.g., CCDs), anderrors in image extraction computation accuracy resulting from theapplication of windows are eliminated the filtering results shouldtheoretically represent the true disparity. Thus, by interpolatingvalues between the pixels in accordance with the sampling theorem withregard to the output of the inverted phase filter discretized on a pixellevel, it is possible to derive the true peak position on a subpixellevel.

(Peak Position (Subpixel-Level Disparity) Detection)

In step S16, peak position detection section 104 detects peak positionsbased on the filtering results, and computes the subpixel-leveldisparity (i.e., the displacement of the peak positions in the X-axisdirection) between the target image and the reference image. Peakposition detection section 104 then outputs the computation result tohigh-accuracy filter section 105.

(Window Function Shift)

In step S17 window function shift section 411 forms a shifted windowfunction for subpixel-level estimation by shifting the window functionfor subpixel-level estimation used for the extraction of the unitpartial reference image for subpixel estimation at filter section 103 byan amount corresponding to the subpixel-level disparity between thetarget image and the reference image computed at peak position detectionsection 104. The shifted window function for subpixel-level estimationis used at image extraction section 412 to extract the unit partialreference image for subpixel estimation. Window function shift section411 outputs the shifted window function for subpixel-level estimation toimage extraction section 412. Thus, the shifted window function forsubpixel-level estimation set in image extraction section 412.

(Extraction Process for Unit Partial Target Image for SubpixelEstimation)

In step S18, image extraction section 412 extracts the unit partialtarget image for subpixel estimation from the unit partial target imageusing the window function for subpixel-level estimation. The extractionprocess in step S18 is similar to the process in step S12.

(Extraction Process for Unit Partial Reference Image for SubpixelEstimation)

In step S19, using the shifted window function for subpixel-levelestimation, image extraction section 412 extracts a unit partialreference image for subpixel estimation which is based on the positionof the unit partial reference image whose level of match identified instep S7 is the greatest. The extraction process in step S19 is generallythe same as the process in step S13, but uses a different windowfunction to extract the unit partial reference image for subpixelestimation. Specifically, the window functions used in these two steps(i.e., the window function for subpixel-level estimation and the shiftedwindow function for subpixel-level estimation) are offset by a gapcorresponding to the disparity detected in step S16.

(Inverted Phase Filter Coefficient Computation Process)

In step S20, filtering section 413 computes an inverted phase filtercoefficient based on the unit partial target image for subpixelestimation. The computation process in step S20 is the same as theprocess in step S14.

(Filtering Process)

In step S21, filtering section 413 performs filtering on the unitpartial reference image for subpixel estimation using the inverted phasefilter coefficient computed in step S20, and outputs the filteringresults to peak position detection section 106.

(Peak Position (Subpixel-Level Disparity) Detection)

In step S22, peak position detection section 106 detects peak positionsbased on the filtering results, and computes the subpixel-leveldisparity (i.e., the displacement of the peak positions in the X-axisdirection) between the target image and the reference image.

The concept of a process at high-accuracy filter section 105 will now bedescribed.

FIG. 8 is a diagram illustrating the concept of a process at ahigh-accuracy filter section. With respect to FIG. 8, as window functionw(m) for subpixel-level estimation for the target image, a Hanningwindow that is so sized that the vertical axis is one pixel and thehorizontal axis is “K-J” pixels is used. m is an integer equal to orgreater than J but equal to or less than K. Window function w(m) forsubpixel-level estimation is set with target point (xa, ya) at thecenter. Thus, an image whose vertical axis size is one pixel and whosehorizontal axis size is “K-J” pixels is extracted, as the unit partialtarget image for subpixel estimation, with target point (xa, ya) at thecenter. f′(m) represents the intensity value of the unit partial targetimage for subpixel estimation.

On the other hand, as shifted window function w(m) for subpixel-levelestimation for the reference image, a Hanning window which is so sizedthat the vertical axis is one pixel and the horizontal axis is “K-J”pixels, and which is obtained by shifting window function w(m) forsubpixel-level estimation by an amount corresponding to thesubpixel-level disparity computed at peak position detection section 104is used. Shifted window function w(m) for subpixel-level estimation isset with corresponding point (xa+n, ya) at the center. Thus, an imagewhose vertical axis size is one pixel and whose horizontal axis size is“K-J” pixels is extracted, as the unit partial reference image forsubpixel estimation, with corresponding point (xa+n, ya) at the center.

FIGS. 9A through 9E are charts illustrating the concept of a process bywindow function shift section 411. FIG. 9A shows the intensity signal ofthe target image across the extracted range (e.g., 15 pixels) and theintensity signal of the reference image across the extracted range. Inother words, FIG. 9A plots the intensity value at each pixel withrespect to 15-pixels'-worth of both the target image and the referenceimage. FIG. 9B shows a widow function for subpixel-level estimation forextracting the unit partial target image for subpixel estimation and theunit partial reference image for subpixel estimation at filter section103.

FIG. 9C shows the results of applying the window function forsubpixel-level estimation shown in FIG. 9B to the target image andreference image shown in FIG. 9A. Specifically, FIG. 9C shows intensitysignals wherein the effects of discontinuity at their end points havebeen reduced through the application of the window function forsubpixel-level estimation. FIG. 9D shows, with respect to high-accuracyfilter section 105, a window function for subpixel-level estimation forextracting the target image for subpixel estimation and a shifted windowfunction for subpixel-level estimation for extracting the referenceimage for subpixel estimation. In this case, the disparity between thewindow function for subpixel-level estimation and the shifted windowfunction for subpixel-level estimation (i.e., the shift amount impartedto the shifted window function for subpixel-level estimation)corresponds to the phase difference between, with respect to FIG. 9C,the results of applying the window function for subpixel-levelestimation to the target image (the target image for subpixel estimationoutputted by image extraction section 402) and the results of applyingthe window function for subpixel-level estimation to the reference image(the reference image for subpixel estimation outputted by imageextraction section 402).

FIG. 9E shows the results of applying the window function forsubpixel-level estimation and the shifted window function forsubpixel-level estimation to the target image and the reference image,respectively, shown in FIG. 9A (the target image for subpixel estimationoutputted by image extraction section 402 and the target image forsubpixel estimation outputted by image extraction section 412).

FIG. 10 is a chart illustrating an effect of a process by windowfunction shift section 411. With respect to FIG. 10, the horizontal axisrepresents the period of the sin function used as a window function, andthe vertical axis represents differences between true values of parallaxand computed subpixel-level disparities between the target image and thereference image.

Curve 1001 formed by the “♦” marks plotted in FIG. 10 shows the resultsof peak position detection performed based on the pixel data sequenceshown in FIG. 9C of the unit partial target image for subpixelestimation extracted from the target image by applying the windowfunction for subpixel-level estimation, and the pixel data sequence ofthe unit partial reference image for subpixel estimation extracted fromthe reference image by applying the window function for subpixel-levelestimation.

On the other hand, curve 1002 formed by the “▴” marks plotted in FIG. 10shows the results of peak position detection. performed based on thepixel data sequence shown in FIG. 9E of the unit partial target imagefor subpixel estimation extracted from the target image by applying thewindow function for subpixel-level estimation, and the pixel datasequence of the unit partial reference image for subpixel estimationextracted from the reference image by applying the shifted windowfunction for subpixel-level estimation.

As can be seen from curve 1001 and curve 1002 in FIG. 10, when thewindow function for subpixel-level estimation is not shifted, thedifferences (errors) between the computed subpixel-level disparities andthe true values are greater, whereas when the window function forsubpixel-level estimation is shifted, the differences (errors) betweenthe computed subpixel-level disparities and the true values aregenerally close to 0. In other words, by shifting the window functionfor subpixel-level estimation, it is possible to improve the accuracy ofthe computed subpixel-level disparity.

The reason that there thus are differences in the errors relative to thetrue values between cases where the window function for subpixel-levelestimation is not shifted and cases where the window function forsubpixel-level estimation is shifted is as follows.

The target image and reference image as shown in FIG. 9A are image datasequences that are referenced based on “disparity n” computed at imagematching section 102. Between “disparity n” computed at image matchingsection 102 and the true values, there exist a maximum error of 0.5pixels and a minimum error of 0 pixels. If the same window function forsubpixel-level estimation such as that shown in FIG. 9B is used on boththe target image and the reference image (i.e., if the shift intervalbetween the window function used for the target image and the windowfunction used for the reference image is zero) when the error isgreatest (i.e., when the error is 0.5 pixels), a subpixel-leveldisparity would be computed in such a manner that it is more greatlyaffected by the fact the phase difference between the window functionsthat are used for the target image and the reference image is 0 pixelsthan it is by the 0.5-pixel difference relative to the true value.Consequently, as shown in FIG. 10, there occurs a difference (error)between the computed subpixel-level disparity and the true value.

On the other hand, when a pair including a window function forsubpixel-level estimation and a shifted window function forsubpixel-level estimation, to which is imparted a shift intervalcorresponding to the error between “disparity n” and the true value, isused (see FIG. 9D), it is possible to perform image extraction inaccordance with the error between “disparity n” and the true value. Asubpixel-level disparity computed using image data thus extracted iscloser to the true value, thereby enabling a drastic reduction in thedifference (error) relative to the true value as shown in FIG. 10.

The improvement in parallax computation accuracy brought about byshifting the window function for subpixel-level estimation increases asthe error between pixel-level “disparity n” and the true value becomesgreater. The error between pixel-level “disparity n” and the true valueis dependent on the base line length of the camera (the distance betweenthe stereo cameras), the focal length of the lenses, and the distancebetween the camera and the object of interest.

Window function shift section 411 may also normalize the coefficient ofthe shifted window function for subpixel-level estimation after shiftingthe window function for subpixel-level estimation. By normalizing thecoefficient, the accuracy of parallax computation may be furtherimproved. The shifted window function for subpixel-level estimation withits coefficient normalized may be represented by the following equation,for example.

$\begin{matrix}{\left( {{Equation}\mspace{14mu} 3} \right)\mspace{619mu}} & \; \\{{{w(m)} = {\frac{1}{2}\left\{ {{\cos \left( {\pi \frac{m - {\Delta \; d}}{K - J}} \right)} + 1} \right\}}}{c = \frac{\left\{ {{\cos \left( {\pi \frac{m}{K - J}} \right)} + 1} \right\} \left\{ {{\cos \left( {\pi \frac{m}{K - J}} \right)} + 1} \right\}}{\left\{ {{\cos \left( {\pi \frac{m}{K - J}} \right)} + 1} \right\} \left\{ {{\cos \left( {\pi \frac{m - {\Delta \; d}}{K - J}} \right)} + 1} \right\}}}} & \lbrack 3\rbrack\end{matrix}$

The processes in step S18 and step S20 in FIG. 6 overlap with step S12and step S14, and may thus be omitted. In this case, the inverted phasefilter obtained through step S12 and step S14 may be stored in memory(not shown), and this stored inverted phase filter may be used in stepS21. Computations may thus be reduced.

<Ranging Target Region Completion Determination Process>

In step S9, a ranging target region completion determination process isperformed. If there exists an unprocessed region that has not yetundergone the processes of step S1 through step S8, the processes ofstep S1 through step S8 are performed on that unprocessed region.

Thus, according to the present embodiment, with respect to stereo imageprocessing apparatus 100, image extraction section 402 extracts the unitpartial target image for subpixel estimation and the unit partialreference image for subpixel estimation from the target image and thereference image using the window function for subpixel-level estimation.Filtering section 403 reverses the data order of the data sequenceincluding the intensity values of the unit partial target image forsubpixel estimation extracted by image extraction section 402, therebycomputing the inverted phase filter coefficient. Filtering section 403performs filtering on the data sequence including the intensity valuesof the unit partial reference image for subpixel estimation using theinverted phase filter coefficient. Peak position detection section 104computes the subpixel-level disparity based on the peak position withrespect to the filtering results of the filtering section.

Based on the subpixel-level disparity computed by peak positiondetection section 104, window function shift section 411 shifts thewindow function for subpixel-level estimation, thereby forming theshifted window function for subpixel-level estimation and setting itwith respect to image extraction section 412.

Image extraction section 412 extracts the unit partial target image forsubpixel estimation from the target image using the window function forsubpixel-level estimation, and extracts the unit partial reference imagefor subpixel estimation from the reference image using the shiftedwindow function for subpixel-level estimation that has been set bywindow function shift section 411. Filtering section 413 computes theinverted phase filter coefficient by reversing the data order of thedata sequence including the intensity values of the unit partial targetimage for subpixel estimation extracted by image extraction section 412,and performs filtering on the data sequence including the intensityvalues of the unit partial reference image for subpixel estimation usingthe inverted phase filter coefficient. Peak position detection section106 computes the subpixel-level disparity based on the peak position inthe filtering results of the filtering section.

It is thus possible to perform image extraction in accordance with theerror between “disparity n” and the true value. The subpixel-leveldisparity computed using image data thus extracted is closer to the truevalue, consequently enabling an improvement in parallax computationaccuracy.

Embodiment 2

In Embodiment 2, before the high-accuracy filter process, adetermination is made as to whether or not to execute the high-accuracyfilter process, and the high-accuracy filter process is executed inaccordance with this determination result.

[Configuration of Stereo Image Processing Apparatus 900]

FIG. 11 shows a configuration of stereo image processing apparatus 900according to Embodiment 2 of the claimed invention. With respect to FIG.11, stereo image processing apparatus 900 includes high-accuracy filterexecution determination section 901, and output section 902.

Depending on the subpixel-level disparity computed at peak positiondetection section 104, high-accuracy filter execution determinationsection 901 determines whether or not the processes of high-accuracyfilter section 105 and peak position detection section 106 are to beexecuted. The subpixel-level disparity computed at peak positiondetection section 104 assumes some value from zero to a maximum value.The range from zero to a base value is taken to be a “turn-executionregion,” while the range from the base value to the maximum value istaken to be an “execution region.” The base value may be determinedduring operation based on a trade-off between processing time and therequired parallax accuracy. By way of example, if parallax accuracy isnot a significant concern, while there is a need for a shorterprocessing time, the base value may be set at a relatively high value.

High-accuracy filter execution determination section 901 determineswhich of the “non-execution ecution region” and the “execution region”the subpixel-level disparity computed at peak position detection section104 falls in. If it is determined that it falls in the “non-executionregion,” high-accuracy filter execution determination section 901outputs to output section 902 the subpixel-level disparity outputtedfrom peak position detection section 104. On the other hand, if it isdetermined that it falls in the “execution region,” high-accuracy filterexecution determination section 901 outputs to high-accuracy filtersection 105 the subpixel-level disparity outputted from peak positiondetection section 104.

Output section 902 outputs the subpixel-level disparity outputted fromhigh-accuracy filter execution determination section 901 or from peakposition detection section 106.

(Operations of Stereo Image Processing Apparatus 900)

FIG. 12 is a flow chart showing the details of a subpixel-levelcomputation process according to Embodiment 2. In contrast to the flowchart shown in FIG. 6, the flow chart shown in FIG. 12 includes step S31involving a high-accuracy filter execution determination.

In step S31, depending on the subpixel-level disparity computed at peakposition detection section 104, high-accuracy filter executiondetermination section 901 determines whether or not the processes ofhigh-accuracy filter section 105 and peak position detection section 106are to be executed.

If it is determined that they are to be executed (step S31: Yes), theprocesses of step S17 through step S22 are executed.

On the other hand, if it is determined that they are not to be executed(step S31: No), the flow shown in FIG. 12 is terminated.

As mentioned above, the improvement in parallax computation accuracybrought about by shifting the window function for subpixel-levelestimation increases as the error between pixel-level “disparity n” andthe true value increases. Accordingly, through a determination based onthe subpixel-level disparity computed at peak position detection section104, high-accuracy filter execution determination section 901 is able toonly have the processes of high-accuracy filter section 105 and peakposition detection section 106 executed when the improvement in parallaxcomputation accuracy is significant. Since it is thus possible to omithigh-accuracy filter section 105 and peak position detection section 106when the improvement in parallax computation accuracy is small, it ispossible to reduce computations.

Other Embodiments

(1) For the various embodiments above, image extraction section 402 andimage extraction section 412 have been described as distinct functionalsections, as have filtering section 403 and filtering section 413, aswell as peak position detection section 104 and peak position detectionsection 106. However, the claimed invention is by no means limited assuch, and instead, image extraction section 402 and image extractionsection 412 may be provided as a single functional section, as mayfiltering section 403 and filtering section 413, as well as peakposition detection section 104 and peak position detection section 106.

(2) For the embodiments above, the processes of steps S17, S19, S21 andS22 are not limited to being performed once, and may simply be repeateda plurality of times, or they may be repeated until the value of thesubpixel-level disparity detected in step S22 falls to or below a basevalue. This enables a further improvement in parallax detectionaccuracy. In this case, in step S17 in the second and subsequent passes,a window function that is shifted by an amount corresponding to thesubpixel-level disparity detected in the immediately preceding step S22is used as the shifted window function for subpixel-level estimation.

(3) For the embodiments above, descriptions have been provided where thedisparity between the target image and the reference image is detectedroughly at image matching section 102 on a “pixel level,” and where thedisparity is subsequently detected finely at peak position detectionsection 104 on a “subpixel level.” In other words, descriptions havebeen provided with regard to a case where disparity is computed in twostages. However, the claimed invention is by no means limited as such,and is also applicable to cases where disparity is computed directly ona “subpixel level” without performing the “pixel-level” detection.

(4) For the embodiments above, descriptions have been provided taking asan example a case where the claimed invention is configured withhardware. However, the claimed invention may also be realized throughsoftware in cooperation with hardware.

(5) The functional blocks used in the descriptions for the embodimentsabove are typically realized as LSIs, which are integrated circuits.They may be individual chips, or some or all of them may be integratedinto a single chip. Although the term LSI is used above, depending onthe level of integration, they may also he referred to as IC, systemLSI, super LSI, or ultra LSI.

The method of circuit integration is by no means limited to LSI, and mayinstead be realized through dedicated circuits or general-purposeprocessors. Field programmable gate arrays (FPGAs), which areprogrammable after LSI fabrication, or reconfigurable processors, whoseconnections and settings of circuit cells inside the LSI arereconfigurable, may also be used.

Furthermore, should there arise a technique for circuit integration thatreplaces LSI due to advancements in semiconductor technology or throughother derivative techniques, such a technique may naturally be employedto integrate functional blocks. Applications of biotechnology, and/orthe like, are conceivable possibilities.

The disclosure of the specification, drawings, and abstract included inJapanese Patent Application No. 2010-283622, filed on Dec. 20, 2010, isincorporated herein by reference in its entirety.

INDUSTRIAL APPLICABILITY

A stereo image processing apparatus and a stereo image processing methodof the claimed invention are effective in that they enable accurateparallax computation even for objects of small image region sizes in thebase line length direction.

Reference Signs List

100, 900 Stereo image processing apparatus

101 Stereo image acquisition section

102 Image matching section

103 Filter section

104, 106 Peak position detection section

105 High-accuracy filter section.

402, 412 Image extraction section

403, 413 Filtering section

411 Window function shift section

901 High-accuracy filter execution determination section

902 Output section

1. A stereo image processing apparatus that computes a subpixel-leveldisparity between a target image and a reference image that form stereoimages, the stereo image processing apparatus comprising: an extractionsection that extracts a unit partial target image for subpixelestimation from the target image by applying a first window function atan extraction target position, and that extracts a unit partialreference image for subpixel estimation from the reference image byapplying the first window function or a second window function that isset; a computation section that computes the subpixel-level disparitybased on a phase difference between a data sequence including intensityvalues of the extracted unit partial target image for subpixelestimation and a data sequence including intensity values of theextracted unit partial reference image for subpixel estimation; and awindow function setting section that sets, with respect to theextraction section, the second window function, which is formed byshifting the first window function based on the subpixel-level disparitycomputed by the computation section.
 2. The stereo image processingapparatus according to claim 1, wherein the computation section:computes an inverted phase filter coefficient by reversing the dataorder of the data sequence including the intensity values of theextracted unit partial target image for subpixel estimation; performsfiltering on the data sequence including the intensity values of theunit partial reference image for subpixel estimation using the invertedphase filter coefficient; and computes the subpixel-level disparitybased on a peak position in a result of the filtering.
 3. The stereoimage processing apparatus according to claim 1, further comprising adetermination section that determines whether or not a process of thewindow function setting section is to be executed depending on thesubpixel-level disparity computed based on the unit partial target imagefor subpixel estimation and unit partial reference image for subpixelestimation extracted with the first window function.
 4. A stereo imageprocessing method that computes a subpixel-level disparity between atarget image and a reference image that form stereo images, the stereoimage processing method comprising: an extraction step of extracting aunit partial target image for subpixel estimation from the target imageby applying a first window function at an extraction target position,and extracting a unit partial reference image for subpixel estimationfrom the reference image by applying the first window function or asecond window function that is set; a computation step of computing thesubpixel-level disparity based on a phase difference between a datasequence including intensity values of the extracted unit partial targetimage for subpixel estimation and a data sequence including intensityvalues of the extracted unit partial reference image for subpixelestimation; and a window function setting step of forming and settingthe second window function by shifting the first window function basedon the subpixel-level disparity computed through the computation step.